Skip to content

Fix process output truncation at embedded NUL bytes - #253

Merged
bob sira (bobsira) merged 1 commit into
mainfrom
fix/logmonitor-process-nul-buffering
Nov 16, 2026
Merged

bob sira (bobsira) merged 1 commit into
mainfrom
fix/logmonitor-process-nul-buffering

Conversation

@bobsira

Copy link
Copy Markdown
Contributor

Summary

Fixes #249.

  • Append exactly the byte count returned by ReadFile, rather than treating each process-pipe buffer as a NUL-terminated C string.
  • Remove the now-unneeded buffer terminator. Embedded NULs reach the existing JSON/XML formatter, which replaces them with ? without dropping following text or merging independent records.
  • Add JSON/XML regressions using real Windows pipes and the production ReadFromPipe implementation. Cover NULs within lines and at 4096-byte read boundaries, UTF-8 preservation, separate records, and the final partial-line flush when pipe EOF is explicitly signaled. Restore standard-output/global state and close owned test handles.

The reported reproduction now retains two records:

NULL before?after
RC2_AFTER_NULL

Local validation

The two new pipe regressions failed on unchanged main production code before the fix and passed afterward. The original issue was also reproduced with the saved baseline executable.

Validation Result
Full native x64 Release suite 51/51 passed
Full native x86 Release suite 51/51 passed
Full CMake x64 Release suite 51/51 passed
Native x64/x86 Debug process tests 7/7 each
Real-child JSON/XML stress matrix across all three Release builds 240/240 cases; 35,832 records
Windows container matrix 10/10 cases; 590 records
Repository-wide cpplint 2.0.2 All 35 tracked C++ source/header files passed

Stress coverage includes NUL positions, read/write boundaries, valid and invalid UTF-8, stdout/stderr and alternating writes, 1 MiB lines, 5,000-record bursts, deterministic randomized inputs, and child exit codes 0, 37, and 255.

Container validation used process isolation on Server Core ltsc2025 (native x64/x86 and CMake x64) and Nano Server ltsc2022 (native/CMake x64), testing both JSON and XML. The native x86 executable's existing VC++ runtime dependency was supplied app-locally in its fixture. Nano ran as ContainerAdministrator to execute the host bind-mounted fixture; host permissions were not changed.

Native Debug builds still emit the pre-existing STL4038 warning about including <variant> without C++17. No compiler errors occurred. Test-owned containers, processes, and the Log Monitor ETW session were cleaned up.

Scope and follow-ups

Extensive testing confirmed two unchanged baseline limitations, now tracked separately:

This PR does not change shutdown behavior, formatter policy, release versioning, or published release artifacts.

Append the exact ReadFile byte count so embedded NULs reach the existing formatter without truncating data or merging records.

Add JSON/XML pipe regressions covering NUL positions, read boundaries, UTF-8, and final partial-line flushing.

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟢 Approval recommended

No active review issues block approval of the byte-preserving process-output change.

No open issues 🎉

What changed in this PR

Fixes #249 in LogMonitor’s process-output pipeline by preserving every byte returned by pipe reads. Embedded NULs can then reach the existing JSON/XML formatter without truncating text or merging separate records.

Changes:

  • Replace C-string buffering with byte-counted appends and remove the unused buffer terminator.
  • Add JSON/XML regressions covering embedded NULs, 4096-byte read boundaries, UTF-8, separate records, and partial-line flushing at explicit EOF.
  • Restore shared test state and close test-owned pipe handles. Existing CRLF and shutdown behavior remain unchanged.
File Description
LogMonitor/​src/​LogMonitor/​ProcessMonitor.cpp Preserve the complete pipe-read buffer, including embedded NULs.
LogMonitor/​LogMonitorTests/​LogMonitorTests.cpp Add production pipe-reader regressions with output capture and resource cleanup.

Review effort: Max

@bobsira
bob sira (bobsira) merged commit d7583ae into main Nov 16, 2026
10 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[BUG] Embedded NUL bytes truncate and merge child process log lines in JSON and XML

2 participants